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Abstract 

Background: Over 40% of all deaths among children under 5 are neonatal deaths (0-28 days), and this proportion 
is increasing. In 2012, 2.9 million newborns died, with 99% occurring in low- and middle-income countries. Many of 
the countries with the highest neonatal mortality rates globally are currently or have recently been affected by 
complex humanitarian emergencies. Despite the global burden of neonatal morbidity and mortality and risks 
inherent in complex emergency situations, research investments are not commensurate to burden and little is 
known about the epidemiology or best practices for neonatal survival in these settings. 

Methods: We used the Child Health and Nutrition Research Initiative (CHNRI) methodology to prioritize research 
questions on neonatal health in complex humanitarian emergencies. Experts evaluated 35 questions using four 
criteria (answerability, feasibility, relevance, equity) with three subcomponents per criterion. Using SAS 9.2, a 
research prioritization score (RPS) and average expert agreement score (AEA) were calculated for each question. 

Results: Twenty-eight experts evaluated all 35 questions. RPS ranged from 0.846 to 0.679 and the AEA ranged 
from 0.667 to 0.41 1. The top ten research priorities covered a range of issues but generally fell into two 
categories- epidemiologic and programmatic components of neonatal health. The highest ranked question in this 
survey was "What strategies are effective in increasing demand for, and use of skilled attendance?" 

Conclusions: In this study, a diverse group of experts used the CHRNI methodology to systematically identify and 
determine research priorities for neonatal health and survival in complex humanitarian emergencies. The priorities 
included the need to better understand the magnitude of the disease burden and interventions to improve 
neonatal health in complex humanitarian emergencies. The findings from this study will provide guidance to 
researchers and program implementers in neonatal and complex humanitarian fields to engage on the research 
priorities needed to save lives most at risk. 

Keywords: Neonatal, Newborn, Research, Priority-setting, Neonatal infections, Preterm birth, Birth asphyxia, Inequity, 
Epidemiology, Millennium development goals, Complex humanitarian emergency 
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Background 

The push to achieve Millennium Development Goal 
(MDG) 4 — the reduction by two-thirds of the under- 
five mortality rate (U5MR) by 2015 — has drawn greater 
attention to child survival. Over 44% of all deaths among 
children under five are neonatal deaths (0-28 days), and 
this proportion is increasing as deaths decline more rap- 
idly amongst children after the first month of life [1,2]. 
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In 2013, nearly three million newborns died, with almost 
all (99%) occurring in low- and middle-income countries 
[2], Approximately three-quarters of these deaths take 
place in the first week of life with more than a third 
dying within 24 hours of birth [3-5]. The majority of 
these deaths are due to preventable and treatable causes. 
Direct preterm birth complications are the leading cause 
of newborn deaths and the second leading cause of all 
under-five mortality. Preterm birth complications, 
intrapartum-related deaths, and infections account for 
more than 80% of deaths globally [6]. If mothers and 
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newborns receive quality care over two-thirds of new- 
born deaths could be averted worldwide [7]. Newborn 
survival and health in resource-poor settings has re- 
cently emerged on the global health agenda with a re- 
sulting push to define the problem and generate evidence 
on best practices for care [8] but this agenda has not 
translated to emergency settings. 

While under-five mortality rate (U5MR) has been de- 
creasing globally [9-11], progress is slowest in sub- 
Saharan African countries. Many of the countries with 
the highest neonatal mortality rates are currently or have 
recently been affected by complex humanitarian emer- 
gencies (see Table 1) [12] [13-15]. Complex humanitarian 
emergencies are due to conflict, war or civil disturbance, 
natural disasters, food insecurity or other crises that affect 
large civilian populations and result in significant excess 
mortality. These crises are often characterized by popula- 
tion displacement as well as the collapse of basic health 
services and local and national infrastructure that results 
in the need for international assistance as the emergency 
exceeds the capacity of the local government [16]. 

In 2011 it was estimated that 42.5 million people were 
displaced by events such as war and natural disasters 
[17]. Women and children often make up a majority of 
the displaced population and account for a large propor- 
tion of disability, morbidity and mortality among this 
population [18]. The main adverse health consequences 
within this group are due to poor pregnancy outcomes 
that affect both the mother and newborn, and to the in- 
creased risk of sexual violence and exposure to HIV/ 
AIDS [16,19,20]. 

Each humanitarian response situation varies enor- 
mously in how efficiently and equitably health services 
are provided. Many areas of humanitarian response are 
undergoing a process of reform based on operations re- 
search and information exchange on minimum stan- 
dards of care and consistent principles for providing 
health services. For example, as a result of increasing 
knowledge on best practices for neonatal survival, 
particularly in developing countries, this issue has 
recently been emphasized in research and policy state- 
ments as an essential component of humanitarian re- 
sponse [21-23]. While guidelines exist on what to do to 
improve maternal and newborn health [24,25], it is less 
clear on how to implement those practices systematically 
across settings [15]. 

Neonatal health in complex humanitarian emergencies 

Few published studies exist on the burden of neonatal 
health in complex humanitarian emergencies and even 
fewer provide information on improving service delivery 
in these settings. Therefore, a literature review was 
undertaken to understand the epidemiological burden 
of neonatal mortality in complex humanitarian settings 



[Adler A, Amsalu R, Kerber K, Lawn J, Neonatal Mortal- 
ity Literature Review unpublished]. Using 24 specific 
search terms, researchers identified 34 reports from 17 
different sources including published papers, gray lite- 
rature, unpublished work by epidemiologists, and digi- 
tal and hard copy review of documents, Demographic 
Health Surveys (DHS) [26], Multiple Indicator Cluster 
Surveys (MICS 3) [27], and the International Disaster 
Databases [28]. Key findings from this review showed 
most mortality surveys included crude morality rate 
(CMR) and U5MR, with very few surveys identifying 
neonatal mortality rates or causes specific to newborns. 
Additionally, DHS and MICS surveys from conflict- 
affected countries were reviewed to determine if data 
were disaggregated for the portion of the population that 
was internally displaced and found that few countries 
had that information available. The lack of epidemiologic 
information on the causes of neonatal disability, morbid- 
ity, and mortality and effective interventions in complex 
humanitarian emergencies prompted a discussion on the 
need for more data on neonatal health in these settings. 

Child Health and nutrition research Initiative method 

Funding for public health research in complex humani- 
tarian settings, much like many other settings, is limited 
[29]. Given the scarcity of resources, the Child Health 
and Nutrition Research Initiative (CHNRI) developed a 
systematic method to align health research investments 
with the burden of global child health and the potential 
impact of the research by having key stakeholders iden- 
tify priority research questions [30-32]. The CHNRI me- 
thod is a flexible process and includes up to 15 steps 
that enables prioritizing health research investments at 
the institutional, regional, national, international, or glo- 
bal level. The CHNRI methodology has been used to rank 
research priorities for preventing stillbirths [33] and for 
many other subject areas such as intrapartum-related neo- 
natal deaths, newborn infection, preterm and low birth 
rate, zinc deficiency in children, as well as mental health 
and disability [29,34-45]. Using the CHNRI method has 
the additional potential to engage researchers, donors, hu- 
manitarian agencies and health professionals to improve 
dialogue and sustain and expand beyond the research se- 
lection process to form part of a viable public health strat- 
egy [29,37,39-41,46]. 

The purpose of this study was to use an adapted 
CHNRI methodology to prioritize neonatal health re- 
search most likely to reduce newborn related mortality 
in the unique setting of complex humanitarian emer- 
gencies. The purpose was also to facilitate greater open- 
ness, communication and collaboration among partners 
and improve the allocation of research funds for neo- 
natal health research in complex humanitarian emer- 
gencies [40]. 



Table 1 Conflict and refugee status in the 25 countries with the highest neonatal mortality rate (NMR) [2,13,14,17] 


Rank by 

i iiy 1 1 cj i i icui id la i 

mortality rate 


Country 


Neonatal mortality 

nnn hirthc 
idler/ ijUuu live uiiLiia 

(UNICEF 2013) 


UCDP (2011 and 2012) 

any 1-vjiiiiii.l 

Includes any: war and 
minor conflict, non-state 
conflict and one-sided 
violence 


UNESCO (2010) 

1 CJJIJI L UClllllLIUIl 

of conflict* 


UNHCR (mid-2013) 

Mi imhor cofi inooc 

INUI 1 1 JJtrl 1 Cl UUCC3 

from this country 


UNHCR (mid-2013) 

Mumhor rtf rofunooc 
inuiiiuci ui iduytrtrji 

in this country 


UNHCR (mid-2013) 

IDPc in thic 
lurs hi ii Ma 

country 


1 


Mali 


59 


Yes 




1 82,780 


14,425 


353,455 


2 


Sierra Leone 


59 


No 


Post conflict 


5734 


4154 


0 


3 


Guinea-Bissau 


58 


No 




1,162 


7,787 


0 


4 


South Sudan 0 


57 


Yes 


N/A 


102,651 


223,636 


401,433 


5 


Pakistan 


56 


Yes 


Armed Conflict 


46,046 


1,621,525 


975,478 


6 


Ethiopia 


54 


Yes 


Armed Conflict 


73,926 


406,646 


0 


7 


Guinea 


54 


Yes 


Post conflict 


13,956 


1 0,466 


0 


8 


Mozambique 


54 


No 




63 


4,413 


0 


9 


Bangladesh 


54 


No 




9,545 


231,138 


0 


10 


Nepal 


53 


No 


Armed Conflict 


7,323 


51,232 


0 


11 


Nigeria 


52 


Yes 


Non-State 


17,735 


1,849 


0 


12 


Angola 


52 


No 


Post conflict 


16,229 


23,718 


0 


13 


India 


51 


Yes 


Armed Conflict 


1 1 ,784 


1 87,024 


0 


14 


Liberia 


51 


No 


Post conflict 


22,488 


58,852 


0 


15 


Malawi 


50 


No 




275 


6,369 


0 


16 


Somalia 


50 


Yes 


Armed Conflict 


1,130,939 


2,339 


1,122,559 


17 


Afghanistan 


50 


Yes 


Armed Conflict 


2,552,208 


16,866 


5,367 


18 


Niger 


48 


No 




657 


50,424 


0 


19 


Cote d'lvoire 


48 


Yes 


Post conflict 


93,738 


4,025 


24,000 


20 


Democratic Republic of Congo 


47 


Yes 


Armed Conflict 


490,095 


1 83,244 


2,607,407 


21 


Equatorial Guinea 


47 


No 




214 


0 


0 


22 


Timor-Leste 


47 


No 


N/A 


N/A 


N/A 


N/A 


23 


Central African Republic 


47 


Yes 


Extra definitional 


221,577 


1 7,732 


206,000 


24 


Chad 


47 


No 


Armed Conflict 


39,329 


418,451 


90,000 


25 


Burundi 


46 


No 


Armed Conflict 


73,143 


44.034 


78,948 



includes any: war and minor conflict, non-state conflict and one-sided violence. Data from 2011 and 2012. 

^UNESCO report definitions based on battle-related deaths: from Armed Conflict Dataset or the Battle Death Dataset 1990-2008. Conflict-affected means two things: The problems caused by an ongoing or very recent 
conflict and the problems that associated with a post-conflict country. A conflict defined as active will have produced more than 1000 BRD for the period 1999-2008 in addition to more than 200 BRD in any of the 
years 2006, 2007 or 2008. If a conflict meets the former criterion but not the latter we classify it as a post-conflict country. Extra definitional was used to denote that conflict was known to occur in this country but did 
not meet the criteria for each category stated above. 

°South Sudan seceded from the Republic of Sudan in 201 1. IDP figure in South Sudan includes 209,700 people who are in an IDP-like situation. An unknown number of refugees from South Sudan may be included 
under Sudan (500,014) (in absence of separate statistics for both countries). 
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Methods 

Save the Children and the US Centers for Disease Con- 
trol and Prevention (CDC) invited technical experts to 
apply the CHNRI methods to systematically list research 
questions and use standard methods to score them (see 
Appendix 1 and Additional file 1). We applied the fol- 
lowing processes: (i) Defining the research context, ques- 
tions, and criteria for priority setting; (ii) Enlisting experts 
to systematically score the research questions, and; (iii) 
Computing and writing up the results. 

Defining the research context, questions, and criteria for 
priority setting 
Research context 

The research context was defined by space, time, popu- 
lation, and burden of disease. The space we used was 
complex humanitarian emergencies, defined as acute or 
chronic situations of conflict, war or civil disturbance, 
natural disasters, food insecurity or other crises that 
affect large civilian populations, result in significant ex- 
cess mortality, and are beyond the capacity of the local 
government to cope [16]. Time for the context of the re- 
search was defined as present day to the year 2020. The 
burden of disease was all cause mortality and disability 
during the neonatal period. The population of interest 
was internally displaced persons (IDP), refugees, and 
communities affected by complex humanitarian emer- 
gencies in low and middle income countries. 

Research questions 

Global experts in neonatal health working on Save the 
Children's Saving Newborn Lives program compiled a 
list of 97 research questions, some of which were used 
in previous research prioritization exercises for non- 
emergency settings. Using a research pipeline model that 
promotes a global perspective and intent to scale-up in 
both low and high-income settings [47], the research 
questions were categorized into descriptive epidemiology 
(describing the situation and understanding the determi- 
nants), leading to development (improving existing in- 
terventions by increasing deliverability or reducing cost), 
delivery (health policy and implementation research and 
scale up), and discovery (new and basic science concepts 
or technologies). The 97 neonatal research questions 
were reviewed by the experts to remove duplication as 
well as ensure appropriate applicability and relevance to 
complex humanitarian settings. A final list of 35 ques- 
tions was used for the prioritization exercise. The final 
questions were categorized into descriptive (n = 9), de- 
velopment (n = 8), and delivery (n = 18). Based on expert 
selection, there were no discovery questions included in 
the final list. 



Criterion selection 

Criteria were agreed upon for use in prioritizing the re- 
search questions. These criteria were chosen based on a 
standard methodology used in previous prioritization 
exercises developed by the Child Health and Nutrition 
Research Initiative (CHNRI) and used for multiple exer- 
cises in research priority setting for varied subjects [37]. 
Each of the 4 criterion had three subcomponents (see 
Additional file 2). 

1. Answerability: likelihood that research would lead to 
new knowledge in an ethical way; 

2. Feasibility: likelihood that research can be conducted 
cost-efficiently and leads to generalizable new 
knowledge; 

3. Relevance: likelihood that research would address 
important condition and critical gap in knowledge 
and could be readily translated to inform policies 
and programs 

4. Equity: likelihood that the proposed epidemiological 
research would have positive impact on equity and 
local ownership. 

Enlisting experts to systematically score the research 

An inclusive approach was taken and participation was 
widely solicited from persons with subject matter expert- 
ise (neonatal, child health, reproductive health and com- 
plex humanitarian emergencies), including both field 
staff and those at head offices. Initially, an email was 
sent to members of the Inter-agency Working Group 
(IAWG) on Reproductive Health in Crises sub-working 
group on Research, Data and Heath Information Sys- 
tems and the general IAWG list serve [48]. In addition, 
researchers were advised to extend the invitation to 
others who may have appropriate technical expertise. 
Lastly, information about the study was presented at the 
IAWG 13th annual workshop where all attendees who 
had the appropriate expertise were encouraged to par- 
ticipate and they were followed up with an email invita- 
tion [49]. This activity took place between October 14 
and December 10, 2011. 

Participants were asked to complete the ranking by 
completing either a web-based survey or spreadsheet re- 
ceived through e-mail from research coordinator. Within 
the email and web link, participants were provided with 
the definitions regarding the space, time and disease 
burden of interest. The instructions asked participants 
to prioritize epidemiological methods for measurement 
and implementation research questions to achieve a re- 
duction in the number of newborn deaths in complex 
humanitarian emergencies. 

For each of the 35 research questions, participants 
were asked to review all four criteria. Each criterion had 
three subcomponents (see Additional file 2). Participants 
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were asked to select "YES" or "NO" to the best of their 
knowledge. The written instructions in the spreadsheet 
or survey stated that if they felt they understood the 
question well and possessed knowledge to answer it, but 
if they felt the answer wasn't a clear "YES" or "NO" to 
enter the option "YES or NO". In the cases where partic- 
ipants felt they did not have enough knowledge or infor- 
mation to answer a question, they were instructed to 
select "DON'T KNOW" rather than guess. 

Data analysis 

Responses were scored: "YES" (1 point); "NO" (0 points); 
"YES or NO" (0.5 points); and "DON'T KNOW" (miss- 
ing data). If the survey was not done via the web, the 
completed worksheets were returned to the research co- 
ordinator. Using SAS 9.2 (SAS Institute, Cary, NC). An 
overall mean research prioritization score (RPS) was cal- 
culated by taking the average of the four criteria [38]. 
For each research criterion, the average score was com- 
puted by taking the average of the subcomponent scores. 
Questions were ranked in order of the highest RPS. 

The overall research priority score (RPS) was com- 
puted as the mean of the scores for the four criteria, ac- 
cording to the input from the experts, according to the 
formula: 

mean CI + meanCI + meand + meanCA 
4 

C designates the scores for relevant criteria. 

As a measure of agreement among scorers, the Aver- 
age Expert Agreement (AEA) [37,38] scores were com- 
puted for each research question. The AEA was defined 
as the average of the highest proportion of the 12 sub- 
components among scorers for each of the 28 questions 
asked. This gave us an indication of the most frequent 
response among scorers, for example an AEA measure 
of 0.70 would say that on average 70% agreed on the 
most frequent response. The AEA was computed for 
each scored research question as: 

scorers who provided most frequent response 
12 N of all scorers 

We conducted a sensitivity analysis to determine if the 
priorities changed when information was included from 
participants that completed the survey and those that 
were incomplete. We also reviewed all questions to de- 
termine the proportion of those who answered "yes/no" 
or "don't know" for all subcomponents of an individual 
question. Additional information on methods is available 
in web appendices (A-D). 

The Center for Global Health at the US Centers for Dis- 
ease Control and Prevention reviewed the protocol and 



determined that the activity was not human subject's re- 
search and the primary intent was public health response. 

Funding 

No specific funding was received for this study and 
the time of the experts was covered by their respec- 
tive institutions. 

Results 

Thirty-eight experts participated and 28 experts com- 
pleted the survey. Surveys were completed from October 
17 until December 8, 2011. Experts were from UNHCR, 
UNICEF, UNFPA, Save the Children, the US government 
- CDC, the European Union Humanitarian Aid and Civil 
Protection- (ECHO), three academic institutions and 
nine non-governmental international organizations. 

Research prioritization scores (RPS) ranged from 0.679 
to 0.846; scores closer to 1.0 were considered a higher 
priority (see Table 2). Average expert agreement (AEA) 
ranged from 0.41 lto 0.667. Table 2 presents the priority 
questions ranked by RPS and the correlating AEA for 
those who completed the survey. The mean score for 
each criterion is also presented. 

The majority of questions had only one or two respon- 
dents that selected "don't know" for all components of a 
question. There was one question where eight respon- 
dents (26%) selected "don't know" for all subcomponents; 
"What is the attribute of clean delivery kit distribution 
in decision making process for home or facility based 
delivery?" 

The results were similar for those who completed the 
survey and those that did not finish the entire survey 
(n = 10). The ranking, based on RPS, changed for only 
a few of the questions in the mean score. The overall 
correlation was very high. The top 5 questions remained 
in the 5 priority questions for both those that completed 
the survey and those that did not. 

The top 10 research questions covered a range of is- 
sues along the research continuum. Reflecting the lack 
of basic data to describe the extend of the problem of 
neonatal morbidity and mortality in these settings, four 
questions dealt with measurement -namely, determin- 
ation of additional burden of neonatal mortality in emer- 
gency situations, pregnancy surveillance to measure 
neonatal mortality, verbal autopsy to capture causes of 
neonatal mortality and risk factors for neonatal sepsis. 
The programmatic priorities included a focus on safe 
birth, such as increasing demand for skilled attendance, 
coverage of clean birth practices, and facility delivery up- 
take. Additionally, two of the programmatic questions 
focused on the specific components rather than general 
care-specifically, hand washing and prevention of con- 
genital syphilis. Only one development question was 
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Table 2 Survey completers' priority ranked questions (n = 28), research priority scores, average expert agreement and 
mean criterion scores 

Research Research Priority Average Expert Answerability Feasibility Relevance Equity 
type* Score (RPS) Agreement (AEA) 



Rank Question 



1 What strategies are effective in DEL 
increasing demand for, and use of 

skilled attendance? 

2 What is the feasibility, effectiveness and DEL 
cost of approaches to increase 

coverage of clean delivery practices in 
facilities and in homes? 

3 What is the additional burden of DES 
neonatal mortality in different 

emergency situations (e.g. conflict, 
acute vs. protracted, natural disaster)? 

4 Can simplified pregnancy surveillance at DES 
community level be used to measure 
neonatal mortality? 

5 Can simplified verbal autopsy tools be DES 
adapted for use in emergency settings 

to capture the main causes of neonata 
mortality? 

6 Develop and validate strategies to DEV 
identify preterm babies at community 

level by CHWs and family members 

7 Which risk factors for neonatal sepsis DES 
can be identified in emergency settings 

and these mothers and babies given 
extra support? E.g. low birth weight, 
short gestational age, unhygienic 
delivery, skin and umbilical cord care, 
hypothermia, poor feeding practices. 

8 Can pregnancy surveillance at DEL 
community level contribute to 

increased uptake of facility-based 
delivery? 

9 What is the feasibility, effectiveness and DEL 
cost of different approaches to increase 
coverage of syphilis screening in 

pregnancy, treatment and partner 
treatment? 

10 What is the feasibility, effectiveness and DEL 
cost of different approaches to promote 
handwashing of caregivers? 

1 1 What is the incidence of neonatal DES 
sepsis in emergency settings? 

12 What is the relative proportion of death DES 
in the neonatal period to other causes 

of child mortality in emergency 
settings? 

13 What is the feasibility, effectiveness and DEL 
cost of approaches to increase and/or 
maintain tetanus toxoid coverage? 

14 Can use of perinatal audit reduce the DEL 
incidence of adverse outcomes related 

in acute intrapartum events? 

15 What is the feasibility, effectiveness and DEV 
cost of different approaches to promote 
hygienic cord and skin care? 

16 Evaluate ways to provide thermal care DEV 
and feeding for the very preterm baby 

at or close to home 



0.846 



0.841 



0.833 



0.830 



0.828 



0.826 



0.818 



0.816 



0.814 



0.812 



0.803 



0.794 



0.791 



0.785 



0.783 



0.780 



0.643 



0.616 



0.667 



0.616 



0.661 



0.610 



0.634 



0.837 



0.610 



0.568 



0.610 



0.557 



0.500 



0.548 



0.512 



0.824 



0.827 o.; 



0.823 0.", 



0.870 



0.852 



0.820 0.774 0.863 0.5 



0.836 0.755 0.883 0.£ 



0.799 0.857 0.797 0.867 



0.825 0.872 



0.859 0.855 



0.797 0.765 0.841 0.E 



0.866 0.815 0.790 0.793 



0.827 0.801 0.812 0.814 



0.862 0.835 0.743 0.807 



0.619 0.788 0.764 0.802 0.861 

0.610 0.783 0.784 0.755 0.853 



0.828 0.773 0.679 0.885 



0.805 0.795 0.801 0.740 



0.808 0.782 0.782 0.762 



0.798 0.766 0.744 0.812 
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Table 2 Survey completers' priority ranked questions (n = 28), research priority scores, average expert agreement and 
mean criterion scores (Continued) 



1 7 What is the feasibility, effectiveness and 
cost of a scheme of routine home visits 
for initiation of supportive practices, 
detection of illness and newborn 
survival? 

18 What is the feasibility, costs and 
effectiveness of setting up newborn 
care corners in mobile clinics, first 
referral units and district hospitals? 

19 Can simpler clinical algorithms 
(recognition and management) be 
developed and validated for babies 
who require resuscitation at birth, and 
does this increase met need for 
resuscitation at birth? 

20 Safety, feasibility and effectiveness and 
cost of managing severe neonatal 
infections at or close to home (e.g. 
requiring injectable antibiotics) 

21 What is the incidence, causes and 
outcomes of umbilical and skin 
infections among newborns in 
emergency settings? 

22 What is the feasibility and effectiveness 
of approaches to improve aseptic 
practices in labour rooms, maternity, 
paediatric wards and nurseries? 

23 What is the feasibility, effectiveness and 
cost of approaches to increase 
coverage of antibiotics for prolonged 
rupture of membranes? 

24 What is the relative proportion of 
neonatal infections that are pneumonia, 
sepsis, meningitis and are there reliable 
clinical markers/combination of markers 
to distinguish these conditions? 

25 What is the feasibility and effectiveness 
of approaches to improve quality of 
care in hospitals? 

26 Can introduction of Doppler increase 
the use of partograph to monitor labor? 

27 What is the attribute of clean delivery 
kit distribution in decision making 
process for home or facility based 
delivery? 

28 What is the additional burden of 
stillbirth in different emergency 
situations (e.g. conflict, acute vs 
protracted, natural disaster)? 

29 Can low-cost, robust, simple fetal heart 
monitors be developed and tested that 
are more user-friendly than the Pinard? 
Does use of such a device improve fetal 
heart rate monitoring and reduce 
intrapartum stillbirth and asphyxia- 
related outcomes? 

30 Can UTI screening in pregnancy be 
reduced in cost and made more feasible 
so can be used in lower levels of care? 



DEL 0.771 0.539 

DEL 0.769 0.536 

DEL 0.767 0.539 

DEV 0.763 0.530 

DES 0.762 0.521 

DEL 0.753 0.533 

DEL 0.752 0.494 

DES 0.739 0.521 

DEL 0.739 0.479 

DEL 0.724 0.461 

DEL 0.713 0.429 

DES 0.706 0.515 

DEV 0.700 0.443 

DEV 0.698 0.438 



0.799 0.747 0.743 0.795 

0.790 0.725 0.823 0.739 

0.669 0.785 0.809 0.806 

0.807 0.590 0.836 0.820 

0.789 0.703 0.701 0.855 

0.747 0.754 0.779 0.731 

0.779 0.716 0.738 0.776 

0.686 0.720 0.734 0.815 

0.706 0.729 0.752 0.768 

0.829 0.736 0.655 0.675 

0.729 0.695 0.677 0.751 

0.725 0.604 0.785 0.709 

0.718 0.689 0.770 0.622 

0.726 0.653 0.663 0.751 



Morof et ah Conflict and Health 2014, 8:8 
http://www.conflictandhealth.eom/content/8/1/8 



Page 8 of 12 



Table 2 Survey completers' priority ranked questions (n = 28), research priority scores, average expert agreement and 
mean criterion scores (Continued) 



31 


Can simpler, cheaper technology be 
developed to improve supportive care 
of neonates who require oxygen (such 
as robust pulse oximeter, oxygen 
condensers, low cost CPAP etc.) and 
does this reduce deaths, improve 
outcomes? 


DEV 


0.696 


0.458 


0.734 


0.617 


0.760 


0.675 


32 


How can diagnostic facilities in health 
facilities for identification of neonatal 
sepsis be improved? 


DEL 


0.696 


0.461 


0.662 


0.682 


0.758 


0.684 


33 


What is the feasibility, effectiveness and 
cost of different approaches to promote 
prompt care seeking for illness from an 
appropriate provider? 


DEL 


0.691 


0.426 


0.699 


0.614 


0.705 


0.744 


34 


Identification of new interventions to 
prevent transmission of infections 
during childbirth, e.g. chlorhexidine 
vaginal douche, immune modulators 
like zinc to mothers. 


DEV 


0.679 


0.440 


0.678 


0.674 


0.692 


0.672 


35 


Evaluate different methods of behavior 
change that overcome harmful 
practices and promote positive cultura 
and social norms 


DEL 


0.679 


0.411 


0.751 


0.585 


0.681 


0.699 



•Research type: DES = description; DEL = delivery; DEV = development. 



ranked in the top 10-develop strategies to identify pre- 
term babies at the community level. 

Discussion 

To our knowledge, this is the first systematic ranking of 
research priorities for neonatal survival in emergency 
contexts. We focused on research priorities for neonatal 
mortality reduction because newborns are the most vul- 
nerable members of any population, particularly in pop- 
ulations affected in emergencies. The identified priorities 
are also of relevance for women and many also relate to 
older children. 

Four of the top ten priority questions were identified 
under the "description" category, highlighting the lack of 
knowledge about the burden of neonatal mortality and 
morbidity in emergency settings. The need for defining 
the scope of the public health problem prior to imple- 
menting interventions is outlined in the CDC Public 
Health Model, which begins by describing the problem, 
then identifying risk and protective factors, followed by 
developing and testing prevention strategies, and finally 
disseminating prevention strategies and assurance of 
widespread adoption [50]. Description of the problem 
was prioritized here possibly because the burden of the 
problem needs to be better identified before opportun- 
ities for "delivery" and "development" research questions 
can be put forth. 

The highest ranked question was, "What strategies are 
effective in increasing demand for and use of skilled 



attendance?" highlighting the importance of the time of 
birth and the issue of access to providers who have an 
appropriate level of skill. In a recent research priority 
ranking on reduction of stillbirths and preterm deliveries 
at the community level, the highest ranked question 
was: "Evaluate the financial barriers to facility births at 
the community level (user fee exemption, emergency 
loans, conditional cash transfers, transportation vouchers, 
etc.)" [32]. In both the development and humanitarian set- 
tings, the importance of increasing demand for a safe birth 
with trained providers is evident, but is especially challen- 
ging in emergency contexts. Two other questions in the 
top ten priorities relate to care at birth, but with a focus 
on clean birth (number 2 on clean birth kits, and number 
9 on handwashing). Prioritizing hygienic care may be the 
first and most feasible steps towards ensuring clean and 
safe births in these challenging settings with limited skilled 
personnel and many home births. 

The overall high RPS range (0.679 to 0.846) which is 
closer to 1.0 suggests that the majority of questions were 
considered high priority. It is possible that questions 
were reviewed and deemed high priority in part because 
of the initial review of the original list of questions re- 
duced the total number to those only applicable to this 
setting. This range was similar to only one study thus far 
(range 0.56 to 0.86) [32] whereas another study showed 
a broader range (0.25 to 0.90) [34,36,42-44]. Addition- 
ally, similar to other research prioritization exercises, the 
AEA showed a direct positive association with the RPS, 
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indicating that the agreement among experts was greater 
for the top ranking questions than for the lower ranking 
questions. High and low RPS scores represent high levels 
of agreement whereas a AEA closer to the mean value 
represents more disagreement [32,34,36,42-44]. The one 
question, where eight respondents selected "don't know" 
for all subcomponents, "What is the attribute of clean 
delivery kit distribution in decision making process for 
home or facility based delivery?" may have not been 
clear to some respondents. Review of question wording 
with a broader audience to ensure clarity of meaning 
should be conducted in the future. 

The CHNRI methodology provides a systematic way 
to evaluate and prioritize research questions, although 
some limitations exist. This process, while valuable, was 
time consuming for participants. For each of the 35 
questions, 12 subcomponents needed to be assessed. 
This burden may be reflected in the lack of survey com- 
pletion by ten of the invited participants; however, the 
sensitivity analysis revealed the highest ranked priorities 
were consistent among those who finished the survey 
and those who did not. It is also possible that the re- 
spondents most interested in this topic would have com- 
pleted the survey and that respondent bias may exist. 
This exercise was conducted in English only. In the 
future, multiple languages and additional perspectives 
from those currently operating within complex emer- 
gency settings would be valuable to include in the prio- 
ritization. Although affiliations were provided for each 
expert, data on location of work (field-based vs. head- 
quarters) and educational background were not ob- 
tained. All attempts were made to engage field-based 
staff with a broad range of educational backgrounds 
and expertise; however, the success of these efforts 
may have been limited given the challenge of acces- 
sing field-based staff in acute emergency situations in 
particular. Location of the participant's primary work 
environment, educational background and type of oc- 
cupation should be determined in the future. 

The priority research questions identified in this re- 
view are timely, given the increasing focus in global 
health on newborn survival and health, within the con- 
text of an integrated continuum of care from pregnancy, 
childbirth, the postnatal period and beyond, across all 
levels of service delivery. The global Every Newborn 
Action Plan has identified that in order to accelerate 
progress for newborn health different contexts require 
tailored approaches, with specific attention to pre- 
paredness for, and rapid response to, complex huma- 
nitarian emergencies [51]. Filling the knowledge gaps 
for these settings is an overdue need which will be 
tracked alongside specific milestones and indicators 
through the Every Newborn process and inter-agency 
collaboration. 



Conclusions 

Meeting the needs of pregnant women and newborns in 
complex humanitarian emergencies is challenging but 
not impossible. This ranking of priority research ques- 
tions provides a clear opportunity and need to fill know- 
ledge gaps to meet this challenge. However, the acute 
nature of many emergencies requires systematic capacity 
building before emergencies occur to ensure proven so- 
lutions can be delivered and monitored for impact in 
these high-risk settings. The research priorities identified 
here emphasize an immediate need to focus on both de- 
scriptive epidemiology as well as operations research to 
improve outcomes, especially around the time of birth. 
This exercise has already stimulated discussion and ac- 
tion to address gaps in research by donors, international 
non-governmental organizations (NGOs), national stake- 
holders and researchers. Additional financial and human 
resources dedicated to conducting research in these set- 
tings may bridge the gap and answer critical questions. 
Engaging partners that have expertise in neonatal health 
and those with expertise in emergencies to work to- 
gether is critical to reducing neonatal mortality. A wide 
network of partners in both the development and hu- 
manitarian sphere will be needed to address these know- 
ledge and action gaps in complex humanitarian fields 
and together to improve maternal and neonatal health 
and survival, especially amongst the women and new- 
borns who are most at risk. 

Appendix 1 

The CHNRI methodology for setting priorities in health 
research investments 

STAGE 1: Defining the context and criteria for priority 
setting 

Research priority scores for many research investment 
options may change based on different contexts so 
specifying the context a priori is a critical part of the 
CHNRI process. 

The research context was specified by space, time, 
population, and burden of disease context by Save the 
Children and the US CDC as follows: 

• The space: complex humanitarian emergencies, 
defined as acute or chronic situations of conflict, 
war or civil disturbance, natural disasters, food 
insecurity or other crises that affect large civilian 
populations result in significant excess mortality, 
and are beyond the capacity of the local 
government to cope; 

• Time: from present day to the year 2020; 

• The population of interest: internally displaced 
persons (IDP), refugees, and communities affected 
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by complex humanitarian emergencies in low and 
middle income countries; 
• The burden of disease was all cause mortality and 
disability during the neonatal period. 

STAGE 2: Choice of technical experts 

The research coordinator used an inclusive approach to 
solicit participation from persons with subject matter ex- 
pertise (neonatal, child health, reproductive health and 
complex humanitarian emergencies), including both field 
staff and those at head offices. To appropriately contact 
those with subject matter expertise the following steps 
were taken 

1. Initially, an email was sent to members of the Inter- 
agency Working Group (IAWG) on Reproductive 
Health in Crises sub-working group on Research, 
Data and Heath Information Systems and the gen- 
eral IAWG list serve. 

2. In addition, researchers were advised to extend the 
invitation to others who may have appropriate 
technical expertise. 

3. Lastly, information about the study was presented 
at the IAWG 13th annual workshop where all 
attendees who had the appropriate expertise were 
encouraged to participate and they were followed 
up with an email invitation. 

Every effort was made to invite a mix of people with 
different backgrounds (clinicians, epidemiologists, public 
health experts, program leaders and donors) and from 
different countries (both developed and developing ones) 
and representing headquarters and field staff so that the 
mix contains a diversity of views from the wider research 
community. 

STAGE 3: Scoring of research investment options 

Experts were then asked was to score all research ques- 
tions independently, according to the four agreed cri- 
teria. For each of the 35 research questions and each 
criterion, each expert answered three questions targeted 
to assess the likelihood of the proposed research to comply 
with the priority setting criterion (see Additional file 2). 
This task was completed by all experts. The entire process 
was conducted and completed via e-mail and web survey 
between October 14 and December 10, 2011. 

Further information on methods related to this part of 
the priority-setting process are presented elsewhere in 
greater details. 

STAGE 4: Computations of "research priority scores" 

All the experts answered the questions listed in Additional 
file 2 by "Yes" (1 point) or "No" (0 points). They were also 
allowed to declare an informed but undecided answer 



"Yes or No" (0.5 points) or declare themselves insuffi- 
ciently informed to answer the question "Don't Know" 
(missing input). 

An overall mean research prioritization score (RPS) 
was calculated by taking the average of the four criteria. 
For each research criteria, the average score was com- 
puted by taking the average of the subcomponent scores. 
Questions were ranked in order with the highest RPS. 
They represent a direct measure of collective optimism 
of the scorers. Each of the 35 listed research questions 
received four intermediate scores (each ranging between 
0-1.00). 

Assessment of agreement between scorers 

As a measure of agreement among scorers, the Average 
Expert Agreement (AEA) was defined as the average of 
the highest proportion of the 12 subcomponents among 
scorers for each of the 28 questions asked. The AEA is 
informing us, for an average question, what proportion 
of scorers gave the same most frequent answer. 

"CHNRI methodology has the ability to expose the is- 
sues of greatest agreement and controversy. This allows 
more focused discussion among experts following this 
exercise, and informs the investors and policy makers 
about the amount of controversy that surrounds each re- 
search question. The datasets that CHNRI methodology 
produces are not appropriate for application of the usual 
Kappa agreement statistics" [37]. 

Advantages and limitations of the CHNRI methodology 

The applied CHNRI methodology proved to be helpful 
to systematically list and score a very large number of 
specific research questions, as shown recently in exer- 
cises on research prioritization for preventing stillbirths, 
intrapartum-related neonatal deaths, newborn infection, 
and preterm and low birth rate, zinc deficiency in chil- 
dren, as well as mental health and disability [29,33-45]. 

Additionally, the CHNRI process is systematic, trans- 
parent, that uses an a priori, well-defined context and 
criteria. This process engages multiple stakeholders and 
allows for scoring that limits influence of strong-minded 
individuals on the rest of the scorers and quantitatively 
determines research priorities and degree of agreement 
or disagreement between experts. 

Although the CHNRI methodology offers many advan- 
tages, there are still limitations. Though initial list of 
questions evaluated by the experts represents a system- 
atic determination of potential priorities, there may be 
additional questions not included on this list that war- 
rant consideration. Additionally a concern that has been 
raised previously, is that he result of the CHNRI process 
could represent a biased opinion of a very limited group 
of involved experts [33]. The number of people globally 
who possess enough experience, expertise and knowledge 
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on this neonatal health and complex humanitarian emer- 
gencies to be able to judge a very diverse spectrum of re- 
search questions is rather limited. As this is a limited pool 
of people, we feel reassured that a response from 28 par- 
ticipants falls within the range of other exercises [37,38] 
and represented a diverse group of people. 

Validation of CHNRI methodology 

The CHNRI methodology has been used in many previ- 
ous studies with success. Additional information on the 
validity of the methods can be found in the web appen- 
dices of "Setting Research Priorities to Reduce Almost 
One Million Deaths from Birth Asphyxia by 2015" [37]. 

Additional files 
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